
METHOD FOR DETERMINING THE POSITION OF AN OBJECT IN SPACE 

5 A method for determining the position of an object in space Is described, in 
which measurement characteristics of the object are recorded with an optical 
recording device calibrated to a space coordinate system, and on the basis of 
these measurement characteristics, the position of the object in the space 
coordinate system is determined in an image processing device. Such methods 
10 are employed for instance in production and assembly operations in automatic 
production lines, in which a work operation on a manufactured object of unknown 
position is to be performed using manipulating devices. 

In European Patent Disclosure EP 0 91 1 603 B1 , for optically determining the 
15 position of a rigid body in a three-dimensional coordinate system, it is described 
that each of a selection of at least three lines or edges present on the rigid body, 
the reference of which lines or edges to the coordinate system of the body itself is 
known, be reproduced in a two-dimensional coordinate system, in which the 
position of the two-dimensional coordinates is referred to the 3D coordinate 
20 system, and an allocation of marked points is dispensed with. As a rule, the edges 
or lines are detected by a plurality of cameras; it is also possible to use one 
camera that is movable in a controlled way with respect to the 3D coordinate 
system and to make a plurality of pictures of the lines or edges in succession. 

25 However, this has the disadvantage that either a plurality of cameras must be 
used and calibrated separately, or that determining the position of the object in 
space is time-consuming, since a plurality of images must be made from different 
positions. 

3 0 The object of the present invention is therefore to propose a possible way of 


determining the position of a body in space that allows the position of an object in 
space to be ascertained reliably with fewer cameras or camera positions. 

In a method of the type defined at the outset, this object is essentially attained 
5 with the characteristics of claim 1 . To that end, at least two measurement 

characteristics of the object are detected simultaneously in a recording device and 
used to determine the position of the object. To that end, the measurement 
characteristics whose coordinates in their own coordinate system, particularly an 
object coordinate system, are known or can be ascertained by the image 

10 processor, are reproduced in a two-dimensional coordinate system of the 

recording device. Since the location, defined by the position and orientation, of the 
recording device in the space coordinate system is known along with its imaging 
properties, the image processor is able, using evaluation methods known per se, 
to establish a relationship between the measurement characteristics reproduced in 

15 the 2D coordinate system of the recording device and the space coordinate 
system. From the known relationship among the various measurement 
characteristics to one another in the object coordinate system, a conclusion can 
then be drawn about the location of the measurement characteristics in space that 
are recorded. This makes it possible to determine the position of the object in 

20 space. According to the invention, measurement characteristics can also be 

combined into a plurality of groups and detected in various partial measurements; 
the coordinates of the measurement characteristics in one group relative to one 
another are known, without the coordinates relative to measurement 
characteristics of other groups also being known. By combining the various partial 

25 measurements, the coordinates of the total body can be determined with 
increased accuracy. 

Until now, it was assumed that the optical axes of the measuring instmments 
with which the various measurement characteristics were recorded for determining 
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positions, had to be oriented differently in space in order to enable arriving at 
mathematically stable solutions to the evaluation algorithms. This required that 
each measurement characteristic used to detemnine the position of the object in 
space be recorded from a different camera position. This was achieved either by 
5 many cameras, set up at different sites and each calibrated, and meant 

correspondingly high expense for equipment. Alternatively, the camera could be 
moved successively and in a controlled way into various recording positions. 
However, that led to comparatively slow position determination, since all the 
images had to be evaluated in succession. With the present invention, it has now 

10 been found that it is possible to detect a plurality of measurement characteristics 
simultaneously with one camera and to use them for determining the position in 
space. As a result, the speed of determining the position of an object in space is 
increased considerably. This is especially advantageous if the various 
measurement characteristics on the object that are detected in one camera have 

15 a pronounced spacing that in particular is as great as possible. The attainable 
accuracy depends especially on the spacing of the measurement characteristics 
on the object. Depending on the accuracy desired, a defined spacing can be 
specified as a minimum spacing between two measurement characteristics. If the 
demands for accuracy are not so stringent, however, the method of the invention 

2 0 can also be performed with measurement characteristics located close together. 

If the location of an object in space, that is, its position and orientation, is to be 
ascertained accurately, all six degrees of freedom of the object must be 
determined. This can be achieved according to the invention by evaluating at least 
25 three characteristics from at least one recorded image. Determining the position 
and orientation of an object is accordingly possible by evaluating three 
measurement characteristics, which are identifiable in one or more pictures. It is 
especially advantageous if all the measurement characteristics required for the 
evaluation are detected with a single picture. However, it is also possible 
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according to the invention to detect for instance one of the three required 
measurement characteristics using a second recording device or to detect a 
second recording position of the one recording device. By combining at least two 
characteristics in one recording device, a considerable improvement in terms of 
5 the expense for equipment and/or the evaluation speed is already achieved. 

With only three measurement characteristics, however, no residual error can 
be determined. If four or more measurement characteristics are employed, it is 
additionally possible to determine a residual error after an error correction 

10 calculation. The residual error characterizes the error between the modelled 
geometry and reality and is therefore a standard for the accuracy achieved. 
According to the invention, however, it is advantageous if to detect no more than 
five measurement characteristics simultaneously with one recording device. 
Although the residual error decreases with the number of measurement 

15 characteristics, it has been demonstrated that evaluating more than five 

measurement characteristics in one image, using the known evaluation methods, 
does not lead to further significant improvement in temris of error. To avoid 
unnecessary evaluation work, it is therefore possible to do without a relatively high 
number of measurement characteristics to be evaluated in one image of a 

20 recording device. 

Depending on the intended application, it is possible or sufficient to perform 
the determination of the position of the object using fewer than six degrees of 
freedom. Arbitrary subsets are possible. In a two- or one-dimensional 
25 measurement finding, the number of measurement characteristics required can be 
reduced to two or one, respectively, because of the reduced number of degrees of 
freedom. In principle, it is possible to determine the position of the object in from 
six degrees of freedom to only one degree of freedom. Determining five and six 
degrees of freedom is possible by means of at least three measurement 
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characteristics; determining three and four degrees of freedom is possible by 
means of at least two measurement characteristics; and determining one and two 
degrees of freedom is possible by means of at least one measurement 
characteristic. Detecting only one measurement characteristic in one recording 
5 device can also be done with a recording and evaluation system of the invention 
that is arranged for perfomiing the method of the invention. 

In one feature of the method of the invention, the measurement characteristics 
are in particular marked points, that is, pointlike characteristics that themselves 

10 have no orientation in space. Such an orientation of the measurement 

characteristics in space, in contrast to many methods known from the prior art, is 
not necessary for the use of the method of the invention, and in comparison to 
edges for instance offers the advantage that the position of a marked point in 
space is always unambiguous. With an edge of some other geometric contour, 

15 conversely, it Is initially not known which point of the edge is being accurately 
detected. The method of the invention can equally well be employed with 
measurement characteristics which are embodied as edges (mathematical curves 
or straight lines) or other describable geometric forms; in that case, the evaluation 
expenditure is increased. In the case of an edge, the angle of the edge in space 

20 can be used as an additional peripheral condition, in order to increase the 

sturdiness and stability of the mathematical solution model. The method can also 
be employed according to the invention in contour evaluation using arbitrarily 
many points. 

25 Although the method can be well performed with only a single recording 
device, according to the invention it is also possible for a plurality of recording 
devices to be employed. Both the number and the location of recording devices 
involved in the measurement can be made dependent on the extent and geometry 
of the object to be measured and on the required accuracy of the measurement 
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finding. 

In an advantageous feature, a measurement characteristic can also be 
reproduced in a plurality of recording devices. This generally increases the 
5 accuracy of the overall result by the contribution of this measurement 

characteristic, since as a result, a not absolutely necessary overdetermination is 
achieved and is utilized to increase the accuracy. 

To attain high flexibility, it is possible according to the invention to use one or 
10 more stationary and/or movable recording devices, the movable recording devices 
in particular being movable by means of a manipulating device. It is also possible 
for a plurality of cameras, fixedly connected to one another, to be located on one 
manipulating device. By using a manipulating device, it is also possible to put a 
recording device in various measurement positions in succession and to make 
15 images there. In these moving manipulating systems, the manipulating device 
must repeatedly approach the same positions in space, where the recordings are 
made, or must be able to ascertain the calibrated coordinates of its own position in 
space and transfer it to the image processor. 

20 This can be done for instance by means of three- dimensional calibration of 
the recording device on the manipulating device, so that based on the controlled 
motion of the manipulating device, the position of the recording device on the 
manipulating device is known in every attainable state of motion. It is also possible 
to determine the position of the recording device automatically by measuring 

25 characteristics that have known coordinates and/or by using extemal measuring 
means. In particular, it is conceivable for the moving recording devices to be 
calibrated outside the actual measuring space in which the position of the object is 
to be detennined, the calibration being done by placing a calibration medium 
outside this measurement space. To that end, the recording device to be 
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calibrated can be moved by the manipulating device in such a way that the 
calibrating medium is reproduced by the recording device. From the known 
position of the calibrating medium, the position of the recording device can then 
be ascertained and its calibration can be performed. This has the advantage that 
5 the object to be measured need not be detached before the calibration. Because 
of the above-described, especially advantageous calibration options, it is attained 
that for a movable recording device after a motion of the manipulating device, the 
position of the recording device is determined in the space coordinate system. 
The method of the invention is independent of the calibration method, however, 
10 and can be employed with arbitrary calibrations. 

In an advantageous variant of the invention, the recording device is positioned 
such that between visual rays which strike the recording device and originate at 
different measurement characteristics and are used to determined the position of 

15 the object, a large intermediate angle exists in each case. A large intermediate 
angle according to the invention exists whenever the visual rays originating at 
different measurement characteristics do not extend parallel or substantially 
parallel. Preferably, the recording device is positioned such that the intermediate 
angle, or as many intermediate angles as possible, is or are greater than 

20 approximately 10° each. This does not represent a sharp boundary, since the size 
of the intermediate angle determines the accuracy of the result. If the 
requirements for accuracy are suitably slight, even smaller angles of 5° to 6°, for 
instance, can therefore suffice. This is particulariy true if only a few intermediate 
angles, for instance one or two intermediate angles, are smaller than the preferred 

25 intermediate angle of greater than approximately 8° to 10"". Corresponding angle 
limitations apply as the intermediate angles approach 180°. Even in that case, 
sufficient accuracy can still be attained. It has been found according to the 
invention that it is possible to detect a plurality of measurement characteristics 
with one camera and evaluate them with sufficient accuracy if the angles 
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particulariy between the individual visual rays, originating at these measurement 
characteristics, are sufficiently large. In that case, the various visual rays do not 
extend parallel, or do not extend substantially parallel. The attainable accuracy in 
determining the position becomes greater, the larger the intermediate angles 
5 between visual rays. 

This is also true according to the invention for the case in which only one 
measurement characteristic in the one recording device is detected; then overall a 
plurality of recording positions are employed (a plurality of recording devices or a 

10 plurality of positions of one recording device in succession). Even in this case, the 
orientation of the optical axes of the recording device or recording devices at the 
recording positions is not decisive, as long as the visual rays, used for evaluation, 
of various measurement characteristics have sufficiently large intermediate 
angles. An independent aspect of the present invention is therefore directed to 

15 proposing a method for determining the position of an object in space in which 
measurement characteristics of the object are recorded with at least one optical 
recording device, calibrated to a space coordinate system, and on the basis of 
these measurement characteristics, the position of the object in the space 
coordinate system is determined in an image processing device. The recording 

20 device is positioned such that between visual rays that strike the one recording 
device or the plurality of recording devices, which visual rays originate at different 
measurement characteristics and are used to determine the position of the object, 
a large intermediate angle exists in each case. 

25 In practice, as a rule, sufficient accuracies in position determination are 
attained if the intermediate angles between the visual rays are in particular in a 
range from approximately 10'' to 170°. Depending on the particular application, 
however, a larger or smaller angular range may be appropriate. 
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Optimal accuracy is attained if, depending on the particular application, the 
recording device is positioned and/or arranged such that as large an intermediate 
angle as possible exists in each case. This can for Instance be attained by 
adapting the recording distance and focal length of the lens of the recording 
5 device. To attain the largest possible angles between the visual rays, it is 

necessary to use short-focal-length lenses with wide opening angles. With wide- 
angle lenses of this kind in the recording device, which is formed for instance by a 
camera with a CCD sensor, there are wide variations in changes of scale and 
projective outlines of the object reproduction as a function of the object location 
10 and orientation, and this improves the measurement accuracy. The recording 
device is preferably placed so close to the object that the measurement 
characteristics used for the position determination are still just barely detected by 
the recording device. This makes optimal use of the sensitive area, such as a 
CCD chip, offered by the recording device. 

15 

It is decisive for the use of the method of the invention that the coordinates of 
the measurement characteristics, in the coordinates associated with the object to 
i3e measured, or in other words in the object coordinate system, are known for the 
image evaluation and/or can be ascertained by it. This can be attained according 

20 to the invention by providing that the object data are specified, for instance in the 
form of construction data, with the coordinates of the measurement characteristics 
of the image processor. In a further embodiment of the present invention, it is also 
possible for the coordinates of the measurement characteristics on the objects to 
be learned. To that end, the object, in a plurality of known positions, is recorded 

25 by the recording device. The recorded images are then evaluated accordingly, 

and the coordinates are assigned to the various measurement characteristics. The 
image processor can for instance detect all the possible measurement 
characteristics that are available on the object, so that for determining the position 
of the object in space, many different measurement characteristics are available. 
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In a further development of the method of the invention, the selection of 
measurement characteristics to be detected by a recording device, the position of 
the recording device, and/or the focal length of the recording device is determined 
5 automatically. This is especially attractive in the case of moving recording devices, 
since in them the image excerpt to be recorded from the object can be selected 
such that significant measurement characteristics, especially suitable for 
determining the position of the object, optimally cover the sensitive region of the 
recording device. To that end, the recording device can for instance make a test 

10 image of the object. In this test image, the image processor identifies the 
detectable measurement characteristics. By monitoring different recording 
positions and/or different measurement characteristics used for the evaluation, the 
optimal recording position can then be ascertained automatically. Optionally, the 
focal length of the lens of the recording device can additionally be adapted, if the 

15 focal length of the recording device is automatically adjustable. 

Further advantages, characteristics and possible applications of the present 
invention will become apparent from the ensuing description of exemplary 
embodiments and of the drawings. All the characteristics described and/or shown 
20 in the drawings form the subject of the invention, regardless of how they are 
summarized in the claims and/or the claims dependencies. 

Shown are: 

25 Fig. 1, schematically, the determination of the object position in a space 
coordinate system, with a stationary recording device; 

Fig. 2, schematically, the determination of the object position in a space 
coordinate system, with a moving camera; and 
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Fig. 3, schematically, the determination of the object position in a space 
coordinate system relative to a zero position of the object. 


5 In Fig. 1 , a system according to the invention is shown, with which the method 
for determining the position of an object 1 in space according to the present 
invention can be performed. An object 1 is shown, whose position in space is to 
be determined. There is often a need for this in production and assembly 
operations in automated production lines in which the object 1 is positioned in 

10 space for instance by a conveyor belt or the like. The position of the object 1 is 
then unknown. The goal of position determination is usually to be able to perform 
an installation, gripping or machining process on a manufactured object 1 that is in 
an unknown position. Mounting must as a rule always take place at the same 
reference point on the object 1 . To attain this, if the position of the object 1 is 

15 unknown, the object must be measured, and a manipulating device 2, which is 
meant to bring the part to be mounted to the object 1 , must be made to track it in 
accordance with the measurement finding. The tasks to be performed on the 
object range among others from attaching components, such as inserting a door 
into a vehicle body, or intentionally applying material, such as adhesives or 

20 sealing material, to an object 1 , to performing mounting motions on the object, 
such as screwing together connections, welding mechanically prepositioned 
partial objects, or the like. It may also be intended that the object be grasped in 
order to move it. However, the use of the method of the invention is not limited to 
these examples of its use; instead, it can be used in general in determining the 

25 position of an arbitrary body in space. 


To determine the position of the object 1 in space, the object 1 is recorded 
with a recording device 3, such as an optical camera with a CCD sensor. In this 
process, the object 1 is reproduced with measurement characteristics 4 that are 
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used for evaluating the position of the object 1 In space. These measurement 
characteristics 4 may be marked points, that is, point-shaped features that have 
no orientation in space. Geometric shapes, such as circles, lines, the focal point of 
a particularly defined region, comers or edges of the object, or other 
5 characteristics can be used that can be identified unambiguously in the picture 
made by the recording device 3. If the position of the object 1 in space is not 
known in six degrees of freedom, that is, with regard to position and orientation, 
then at least three measurement characteristics 4 which are spaced apart from 
one another on the object are used for the evaluation and are detected in the one 
10 recording device 3. In principle, however, it is also possible to provide a plurality of 
recording devices 3. 


Recording the measurement characteristics 4 of the object 1 in the recording 
device 3 creates a two-dimensional copy of the object 1 that has the 

15 measurement characteristics 4. In an Image processing device, not shown 
separately but connected to the recording device 3, the image made is then 
processed. Based on the three-dimensional calibration of the recording device 3 in 
the space coordinate system 5, which in particular uses the GPS coordinate 
system, and because of the known imaging properties of the recording device 3, 

2 0 defined points in space can be allocated to the measurement characteristics 4 of 
the object 1 . Since the coordinates of the measurement characteristics 4 are 
simultaneously known relative to one another in the object coordinate system 6, 
the location of the object 1 can be ascertained accurately in terms of both the 
position and the orientation in space. 

25 

This information can be used to control the manipulating device 2 such that 
the object 1 is for instance moved to a predetermined position in space, or that 
items located on the manipulating device 2 that are to be mounted are now 
attached to the object 1 in accurate positions. 
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High accuracy can be attained, despite the evaluation of a plurality of 
measurement characteristics 4 in one recording device 3, whenever the evaluated 
measurement characteristics 4 are far enough apart on the object 1 . In that case, 
5 the intermediate angles between visual rays 7 striking the individual measurement 
characteristics 4 in the recording device are also large enough, so that the 
mathematical evaluation algorithms furnish a stable position finding. The position 
of the recording device 3 is preferably selected such that as many visual rays 7 as 
possible, for the measurement characteristics 4 used for the evaluation, have the 
10 largest possible intermediate angle. It is advantageous if the intermediate angles 
between visual rays 7, for as many measurement characteristics as possible, are 
each larger than approximately lO"". 

Fig. 2 shows a comparable system for performing the method of the invention, 
15 in which the recording device 3 itself is attached to a manipulating device 8. In this 
case, in a first recording position of the recording device 3, which is represented 
by solid lines, some of the measurement characteristics 4 used for the evaluation 
are detected. After that, the recording device 3 is moved to a second recording 
position, which is shown in dashed lines. In this second recording position, further 
20 measurement characteristics 4 are recorded and are used for evaluating the 
position of the object 1 in space. 

The method of the invention can also, as shown in Fig. 3, be used to detect 
the relative displacement 9 of an object 1 in space. The principle employed is the 
25 same as in the method variants described above. By means of a recording device 
3, various measurement characteristics 4 of the object 1 in the various positions in 
space are detected and evaluated by means of an image processing device, not 
separately shown. By comparison of the various locations of the object 1 in space 
at various times, the location of the body relative to a zero position can then be 
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ascertained. 

The advantage of the method of the invention resides in particular in the fact 
that a plurality of measurement characteristics 4, preferably up to five 
5 measurement characteristics 4, can be detected and evaluated simultaneously in 
an evaluation device. This simplifies and speeds up the determination of the 
object location in space considerably in comparison to previously known methods. 
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List of Reference Numerals: 

1 

Object 

5 2 

Manipulating device 

3 

Recording device 

4 

Measurement characteristic 

5 

Space coordinate system 

6 

Object coordinate system 

10 7 

Visual rays 

8 

Manipulating device 

9 

Relative displacement 
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